Models, AI and all other buzz words¶

Peer Herholz (he/him)
Postdoctoral researcher - NeuroDataScience lab at MNI/McGill, UNIQUE
Member - BIDS, ReproNim, Brainhack, Neuromod, OHBM SEA-SIG

@peerherholz

A quick poll…¶

Before we start, let’s do a quick poll that will help to start discussions and bring the point of this lecture and workshop across. We will do this in a live and interactive manner via mentimeter. Thus, please head over to www.mentimeter.com and use the following codes on the top of the page to get to the polls. Important: all answers will be anonymously, so don’t worry.

Question 1¶

Please use the code 62447080.

from IPython.display import IFrame

IFrame(src='https://www.mentimeter.com/s/3e455379770da0beb369c8ddf412aa08/33e28848548e', width=700, height=400)

Question 2¶

Please use the code 2893975.

from IPython.display import IFrame

IFrame(src='https://www.mentimeter.com/s/feb3310ac9a14b5ac89854b3e9bed51b/fac70d5f626b', width=700, height=400)

this is exactly why we’re here…

Aim(s) of this section¶

get to know the “lingo” and basic vocabulary
define important terms
situate core workflow aspects

Outline for this section¶

The definitions
The fellowship of core aspects
The two (or more) parts of each aspect
The return of the complexity
Yes, it’s that “easy”

The definitions¶

Artificial intelligence (AI) is intelligence demonstrated by machines, as opposed to the natural intelligence displayed by humans or animals. Leading AI textbooks define the field as the study of “intelligent agents”: any system that perceives its environment and takes actions that maximize its chance of achieving its goals. Some popular accounts use the term “artificial intelligence” to describe machines that mimic “cognitive” functions that humans associate with the human mind, such as “learning” and “problem solving”, however this definition is rejected by major AI researchers.

https://en.wikipedia.org/wiki/Artificial_intelligence

Machine learning (ML) is the study of computer algorithms that can improve automatically through experience and by the use of data. It is seen as a part of artificial intelligence. Machine learning algorithms build a model based on sample data, known as “training data”, in order to make predictions or decisions without being explicitly programmed to do so. A subset of machine learning is closely related to computational statistics, which focuses on making predictions using computers; but not all machine learning is statistical learning. The study of mathematical optimization delivers methods, theory and application domains to the field of machine learning. Data mining is a related field of study, focusing on exploratory data analysis through unsupervised learning. Some implementations of machine learning use data and neural networks in a way that mimics the working of a biological brain.

https://en.wikipedia.org/wiki/Machine_learning

Deep learning (also known as deep structured learning) is part of a broader family of machine learning methods based on artificial neural networks with representation learning. Learning can be supervised, semi-supervised or unsupervised. Artificial neural networks (ANNs) were inspired by information processing and distributed communication nodes in biological systems. ANNs have various differences from biological brains. Specifically, neural networks tend to be static and symbolic, while the biological brain of most living organisms is dynamic (plastic) and analogue. The adjective “deep” in deep learning refers to the use of multiple layers in the network. Early work showed that a linear perceptron cannot be a universal classifier, but that a network with a nonpolynomial activation function with one hidden layer of unbounded width can.

https://en.wikipedia.org/wiki/Deep_learning

The fellowship of core aspects¶

Term	Definition
Model	A set of parameters that makes a prediction based on a given input. The parameter values are fitted to available data.

The two (or more) parts of each aspect¶

Term

Definition

Input

Data from which we would like to make predictions. For example, results from lab tests (e.g. hematocrit, protein concentrations, response time) to inform a diagnosis (e.g. anemia, Alzheimer’s, Muliple Sclerosis). Data is typically multidimensional, with each sample having multiple values that we think might inform our predictions. Conventionally, the dimensions of data are [number of subjects] x [number of features]

Feature

One dimension of the input data which corresponds to a particular measure. When describing a person, ‘height’, ‘weight’, ‘hair colour’ could be considered as features.

Term	Definition
Labels	True values corresponding to a data sample that we would like to accurately predict given. Also known as ‘target’
Prediction	The output of a model for a given sample. In the ideal case, the prediction should be the sample’s label.

The return of the complexity¶

Term	Definition
Preprocessing	Change the distribution of the input (raw feature vectors) to make it more suitable for models, e.g. via standardization, scaling or transforming.

Term	Definition
Estimator	An instance of a model in `sklearn`. An estimator refers to a common interface in `sklearn` that is used to train and evaluate models.
Complexity	Refers to the number of parameters in a model. A model with 10 parameters is said to be more complex than a model with 3.

Term	Definition
Metric	A function that defines how good a prediction is.

Sounds like a lot, eh? Let’s see how these things look like in action…

Yes, it’s that “easy”¶

Let’s imagine we want to use the functional connectivity between brain regions to predict the age of human participants. Using python and a few of its fantastic packages, here pandas and sklearn, we can make this analysis happen in no time.

At first we get our data:

import urllib.request

url = 'https://www.dropbox.com/s/u3vp2ghrsmkw5po/MAIN2019_BASC064_subsamp_features.npz?dl=1'
urllib.request.urlretrieve(url, 'MAIN2019_BASC064_subsamp_features.npz')

('MAIN2019_BASC064_subsamp_features.npz',
 <http.client.HTTPMessage at 0x7fdd92fa30d0>)

Next, we load our input and inspect it:

import numpy as np

data = np.load('MAIN2019_BASC064_subsamp_features.npz')['a']
data.shape

(90, 2016)

We can also use plotly to easily visualize our input data in an interactive manner:

import plotly.express as px
from IPython.core.display import display, HTML
from plotly.offline import init_notebook_mode, plot

fig = px.imshow(data, labels=dict(x="features", y="participants"), height=800, aspect='None')

fig.update(layout_coloraxis_showscale=False)
init_notebook_mode(connected=True)

#fig.show()

plot(fig, filename = 'input_data.html')
display(HTML('input_data.html'))

Beside the input data we also need our labels:

url = 'https://openneuro.org/crn/datasets/ds000228/snapshots/1.0.0/files/participants.tsv'
urllib.request.urlretrieve(url, 'participants.tsv')

('participants.tsv', <http.client.HTTPMessage at 0x7fdd92f6c040>)

Which we can easily load and check via pandas:

import pandas as pd
labels = pd.read_table('participants.tsv')['Age'][:90]
labels.describe()

count    90.000000
mean      5.697269
std       1.644901
min       3.518138
25%       4.321697
50%       5.450000
75%       7.067500
max       9.960000
Name: Age, dtype: float64

For a better intuition, we’re going to also visualize the labels and their distribution:

fig = px.histogram(labels, marginal='box', template='plotly_white')

fig.update_layout(showlegend=False)
init_notebook_mode(connected=True)

#fig.show()

plot(fig, filename = 'labels.html')
display(HTML('labels.html'))

And we’re ready to create our machine learning analysis pipeline using sklearn within we will scale our input data, train a Support Vector Regression and test its predictive performance. We import the required functions and classes:

from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVR
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

and setup a sklearn pipeline:

pipe = make_pipeline(
...     StandardScaler(),
...     SVR()
... )

After dividing our input and labels into training and test sets:

X_train, X_test, y_train, y_test = train_test_split(data, labels, random_state=0)

we can already fit our machine learning analysis pipeline to our data:

pipe.fit(X_train, y_train)

Pipeline(steps=[('standardscaler', StandardScaler()), ('svr', SVR())])

and test its predictive performance:

r2_score(pipe.predict(X_test), y_test) 

-11.006969224310357

You lied!

_{^{_{^{^{https://c.tenor.com/jXn6jy2JkosAAAAC/the-lies-rage.gif}}}}}

I’m sorry.

_{^{_{^{^{https://media4.giphy.com/media/sS8YbjrTzu4KI/giphy.gif?cid=ecf05e47ygr63lz155vgnqurenhc4hoizy9hhcw8dfxtqi2f&rid=giphy.gif&ct=g}}}}}

The truth is: while it’s very easy (maybe too easy?) to setup and run machine learning analyses, doing it right is definitely not (like a lot of other complex analyses)… There are so many aspects to think about and so many things one can vary that the garden of forking paths becomes tremendously large. Thus, we will spent the next few hours to go through central concepts and components of these analyses, first for “classic” machine learning and then for deep learning.

ML/DL with a focus on Neuroscience - SynAGE workshop